Method and apparatus using a medical imaging head for fluorescent imaging

ABSTRACT

The invention relates to a method for detecting visible and infrared light using a medical imaging system, said medical imaging system comprising a camera module configured to receive a visible light signal and at least one infrared light signal from an object image. The medical imaging system for detecting visible and comprises
         an input for visible light for illuminating a tissue;   an input for excitation light for exciting a fluorescence agent in the tissue;   a camera module configured to receive a visible light signal and at least one infrared light signal from an object image in the tissue. The camera module comprises at least a first, second, and third optical path for directing light from the object image to a first, second, and third filter and sensor combination respectively. In any order, the first, second, and third filters are a green filter, an infrared filter, and a red/blue patterned filter comprising red and blue filters in alternating pattern. Half of the red/blue patterned filter is a red filter and the other half is a blue filter. The green and infrared light are thus sampled at full sensor resolution, for the red and blue light spatial interpolation is required.

FIELD OF THE INVENTION

The present invention relates to a method for detecting visible andinfrared light using a medical imaging system and to said medicalimaging system. The medical imaging system may be used in an open lensconfiguration or an endoscope or a laparoscope.

BACKGROUND ART

Various clinical applications using Infrared (IR) or Near-Infrared (NIR)light have been described in the literature. In such applications, acontrast agent may be used which is introduced into living tissue thatis to be examined, and which is absorbs or emits IR light. In someapplications, there are multiple wavelength ranges of IR light which areof interest.

When a device such as an endoscope is used in a clinical applicationusing IR light, the professional using the endoscope typically alsowants to use visible light camera sensing for eye-hand coordination whenusing the endoscope. The endoscope camera must thus sense red, green,and blue visible light and at least one (N) IR wavelength range. Thecamera must also generate the images at a high enough image frequency,or frame rate, to make reliable eye-hand coordination possible. Aminimal frame rate is typically 60 frames per second.

This poses certain constraints on the camera. Current state of the artcolour cameras are 3CCD image heads, which have a single incoming lightentrance and then split the incoming light into three optical pathsending in three respective filter sensor combinations. Each sensortypically senses one primary colour, e.g. red, green, and blue. Thethree colours are typically projected through a prism at different CCDsbut physically look to the same location (the origin of the incominglight). This way for each location the exact R, G and B pixel values aresensed. In order to also sense IR light, such a camera would need afourth optical path which can be provided by adding a further prism orbeam splitter. However, this increasing “stacking” of prisms and beamsplitters has a problem that in order to have all sensors in its focalplane, the first prism is physically the largest, decreasing whenprogressing, in order to compensate the total focal length andcompensate all sensors.

For measuring visible light, it is also possible to use a single sensor(CCD) combined with a Bayer filter, for example having a 2×2 filterpattern of R-G/G-B patterns. Then for each pixel location, only one ofthe pixel values R, G, and B are sensed. The missing pixels need to beinterpolated, which gives rice to an interpolation error especially incase of fine detail. Furthermore the red and blue pixels are one out offour pixels while green is two out of four of these Bayer filters due tothe fact that red and blue provide less information to the human eye.Effectively, 50% of the pixels in the sensor's pixel grid “sees” greenlight, while only 25% of the pixels receives red light and another 25%of the pixels receives blue light. This specifically results in highinterpolation errors in the red and blue signals due to pixel noisewhich can no longer be filtered out because of the low amount ofneighbouring red or blue pixels. This means that hot or dead pixels aredifficult to cancel and give significantly reduced image performanceresulting in visible artefacts. In order to also measure IR light,either a fourth filter must be added (thus increasing the interpolationerror) or a second optical path must be made (increasing the volume).

Both approaches have drawbacks and show that there is a balance betweenspatial errors due to interpolation and pixel noise when using filtersand volume increase due to adding extra prisms (or other types of beamsplitters) for additional optical paths when not using Bayer filters.

U.S. Pat. No. 8,620,410 discloses a medical imaging system that providessimultaneous rendering of visible light and fluorescent images. It usestwo optical paths, one for visible light which is registered by a(single sensor) video camera and one for infrared which is registered byan infrared camera. A drawback of such a system is that the visiblelight camera, in order to generate a colour image using a single CCDsensor, a Bayer filter is used to split the incoming light once intored, green, and blue components. This reduces the resolution of thevisible light image, which is disadvantageous, specifically inendoscopic or laparoscopic imaging systems.

U.S. Pat. No. 9,173,554 addresses that problem by using three opticalpaths, created by a dichroic prism assembly, which are respectively usedfor red, green and blue light sensing. In order to be able to measure afourth component for infrared fluorescence radiation, the optical pathleading to the red light sensor is used to detect both red and infraredlight, in a time alternating fashion. The light source used in theapplication must also alternatingly emit red and infrared light, in syncwith the sensor. In this manner, the spatial resolution is maintained(no spatial interpolation error). However, now there is a temporalinterpolation effect, since the red and infrared lights are sampled athalf the frame rate of the green and blue lights. The effectivelyreduced frame rate and consequently increased time lag hampers theeye-hand coordination of the person using the endoscope.

Furthermore when lower levels of red or infrared light are to bemeasured, a longer exposure time is required, which can also influencethe real time behaviour of the colour image. This approach also does notallow for any calculations that require simultaneous multiple signalsfrom red and infrared wavelengths. Also adding another IR channelreduces the framerate further since now it must be used to detect Red,IR1, IR2, which results in ⅓th of the max frame time, resulting innoticeable artefacts and thus is a clear limit of this technology.

Furthermore, the red and infrared images cannot both be exactly infocus. Due to a large wavelength difference between red and infrared, atbest either the IR image is sharp and in focus or the R image is sharpand in focus. Due to the switching, the sensors can only be set at thesame exposure time, because switching exposure time would require sensorsettings that influence with the readout timing and exposure time, whichcan no longer overlap and ultimately influence either framerate and/orsensitivity of the system as well as adding a great deal of systemprocessing complexity. Finally, gain needs to be switched heavily todiffer between low signal-strength IR signals and relatively highsignal-strength R signals. This negatively influences either thesensitivity of the IR channel and/or the red channel as well as frametime due to time it takes to apply the sensor settings before startingthe exposure.

It is an object of the invention to provide a practical solution fordetecting both visible light and IR light, which does not requiretime-switching of the received signals.

SUMMARY OF THE INVENTION

The invention provides a method for detecting visible and infrared lightusing a medical imaging system, said medical imaging system comprising acamera module configured to receive visible light and infrared lightfrom an object image, the method comprising:

-   -   exciting a fluorescence agent in a tissue sample with excitation        light;    -   illuminating the tissue sample with visible light;    -   receiving light from the object image, said light comprising        visible light and one or more infrared fluorescence light        originating from the fluorescence agent;    -   splitting the received light into a first, second, and third        optical path;    -   filtering the received light in the first, second, and third        optical path through a first, second and third optical filter,        respectively;    -   detecting the filtered light in the first, second, and third        optical path with a first, second and third sensor, respectively        wherein the first, second, and third filters, in any order, are        a green filter, an infrared filter, and a red/blue patterned        filter comprising red and blue filters in alternating pattern so        that half of the light received by the red/blue patterned filter        goes through a blue filter and half of the light received by the        red/blue patterned filter goes through a red filter.

The invention further provides a medical imaging system for detectingvisible and infrared light, said medical imaging system comprising

-   -   an input for visible light for illuminating a tissue;    -   an input for excitation light for exciting a fluorescence agent        in the tissue;    -   a camera module configured to receive visible light and infrared        light from an object image in the tissue, the camera module        comprising at least a first, second, and third optical path for        directing light from the object image to a first, second, and        third sensor respectively, wherein:    -   the first optical path is provided with a first filter,    -   the second optical path is provided with a second filter, and    -   the third optical path is provided with a third filter,        wherein the first, second, and third filters, in any order, are        a green filter, an infrared filter, and a red/blue patterned        filter comprising red and blue filters in alternating pattern so        that half of the light received by the red/blue patterned filter        goes through a blue filter and half of the light received by the        red/blue patterned filter goes through a red filter.

According to the present invention, the tissue sample may be an in-vivotissue sample or an ex-vivo tissue sample.

As mentioned, in the method and the system, half of the light receivedby the red/blue patterned filter goes through a blue filter and half ofthe light received by the red/blue patterned filter goes through a redfilter. In effect, there is one sensor which receives 100% green light,one sensor which receives 100% infrared light, and one sensor whichreceives 50% red light and 50% blue light, in a manner which makesaccurate interpolation of the missing red and blue pixels possible. Allsensors may operate at the same frame rate, there is no need fortemporal subsampling as in the cited prior art.

In an embodiment, the medical imaging system comprises a dichroic prismassembly configured to receive the light from the object image throughan entrance face of the dichroic prism assembly, comprising at least afirst, second, and third prism, each prism having a respective first,second, and third exit face, wherein:

-   -   the first exit face is provided with the first sensor,    -   the second exit face is provided with the second sensor, and    -   the third exit face is provided with the third sensor.

The invention further provides an open lens system, an endoscope and alaparoscope comprising a medical imaging system, more particularly adichroic prism assembly, as described above.

The invention further provides a medical imaging system comprising atleast one dichroic prism assembly as described above, wherein at leasttwo infrared imaging wavelength (e.g. between 640 and 1000 nm) sensorsand three visible wavelength (e.g. red, green, and blue) sensors areused, wherein each of the sensors are optically pixel to pixel alignedwith a accuracy of better than ⅓th of a pixel.

The imaging system may be used for real-time applications duringsurgery. In an embodiment, the frame rate is at least 50 frames persecond (fps) and the pixels are sampled with at least 8 bits. For bettereye-hand coordination, a frame rate of at least 60 frames per second oreven at least 90 or 100 frames per second is preferred. The bit depth ofthe pixels may be the indicated 8 bits or more, e.g. 10, 12 or 16 bits.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be discussed in more detail below, withreference to the attached drawings, in which:

FIG. 1a schematically shows a medical imaging system according to a anembodiment of the invention;

FIG. 1b schematically shows a method according to an embodiment of theinvention;

FIG. 2 schematically shows a dichroic prism assembly according to anembodiment of the invention;

FIGS. 3a-d schematically show filters for use on sensors of a dichroicprism assembly according to an embodiment of the invention;

FIG. 4 schematically shows a further dichroic prism assembly accordingto an embodiment of the invention;

FIG. 5 schematically shows a dichroic prism assembly according to anembodiment of the invention;

FIGS. 6a-c schematically show an endoscope and/or laparoscope andcabling according to an embodiment of the invention; and

FIGS. 7a-b shows an exemplary pentagonal prism for use in an dichroicprism assembly according to an embodiment of the invention.

DESCRIPTION OF EMBODIMENTS

FIG. 1a schematically shows a medical imaging system according to anembodiment of the invention. The figure shows four optical paths: oneoptical path for transmitting visible and excitation light from thevisible and excitation light sources 13 to a tissue 19, and threeoptical paths for transmitting light (visible and fluorescence light)from the tissue 19 to the towards, respectively, light sensors D3, D4,and D5. Each respective light sensor D3, D4, D5 is provided with afilter F3, F4, and F5. The optical paths can be made using dichroicmirrors M1, M2, M3. For clarity, lenses and other optical instrumentsare omitted from FIG. 1 a.

The filters F3, F4, and F5 will be discussed in reference to FIGS. 3a-dwhich schematically show filters for use on sensors of a dichroic prismassembly according to an embodiment of the invention. It is understoodthat the optical path for transmitting visible and excitation light doesnot need to coincide with the any of the other optical paths and thatthe visible and excitation light can be provided to the tissue through aseparate light channel and/or lenses, e.g. provided outside of a camerahead.

FIG. 1b schematically shows a method for detecting visible and infraredlight using a medical imaging system as shown in FIG. 1a . In step 21, afluorescence agent in a tissue sample is illuminated with excitationlight, causing fluorescence light to be emitted by the agent. Meanwhile,in step 22, the tissue sample is also illuminated with visible light.Light from the object image (the tissue sample) is received in step 3,said light comprising visible light and infrared fluorescence lightoriginating from the fluorescence agent. The received light is splitinto a first, second, and third optical path (step 24), and filteredthrough a first (green), second (infrared) and third (red/blue) opticalfilter, respectively. The filtered light is detected by respectivesensors (steps 25, 26, 27). In step 28, a visible light image andinfrared (fluorescence) image is generated from the detected light. Thevisible light image may be shown separately from the fluorescence image.It is also possible to show the fluorescence image as an overlay on thevisible light image.

FIG. 2 shows a prism assembly comprising prisms P5, P6, and P7 forsplitting light in red, green and blue components towards respectivesensors D3, D4, and D5. The light enters the prism assembly through thearrow indicated in FIG. 2. Between P5 and P6, an optical coating C1 isplaced and between prisms P6 and P7 an optical coating C2 is placed,each optical coating C1 and C2 having a different reflectance andwavelength sensitivity. At C1, the incoming beam I is partiallyreflected back to the same face of the prism as through which the lightentered (beam J). At that same face, the beam, now labelled K, is onceagain reflected towards filter F3 and sensor D3. The reflection from Jto K is an internal reflection. Thus, filter F3 and sensor D3 receivelight reflected by coating C1, and in analogue fashion filter F4 andsensor D4 receive light from beam L reflected by coating S2 (beams M andN), and filter F5 and sensor D5 receives light from beam O that hastraversed the prism unhindered.

FIG. 3a schematically shows a patterned filter 20 according to theinvention, in particular with an array of red and blue filters in analternating pattern. The pattern 20 consist of groups of 2×2 pixelswhich are filtered for one particular colour. The groups labelled 21 and22 are intermixed in a checkboard pattern. Pixels in group 21 have a redfilter, and pixels in group 22 have a blue filter. For example the topleft group 21 has red pixels R1, R2, R3, R4 in a 2×2 grid and theneighbouring group 22 has blue filtered pixels B1, B2, B3, B4 in a 2×2grid.

FIG. 3b schematically shows a filter 23 comprising only green filters.For comparison with FIGS. 3a and 3d , the pixels are still shown asdivided in 2×2 groups, but in fact there is a single pixel grid with thelight received at each pixel being filtered with a green filter. Thepixels are therefore labelled in ascending order going from left toright and from top to bottom as G1, G2, G3, etc.

FIG. 3c schematically shows a filter 25 comprising only IR filters.Again, for clarity, the pixels are still shown as divided in 2×2 groups.Now each pixel is filtered with an IR filter. As in FIG. 3b , the pixelsare labelled in ascending order going from left to right and from top tobottom as I1, I2, I3, etc.

FIG. 3d schematically shows a patterned filter 27 comprising twodifferent types of IR filters. As in FIG. 3a , the pixels are divided ingroups of 2×2 pixels and each group has a specific filter. Groups 28have a first IR filter and groups 29 have a second IR filter. The pixelsin the first groups 28 are labelled I1, I, I3, I4, etc., while thepixels in the second groups 29 are labelled J1, J2, J3, J4. This type offilter can be used for applications where more than one wavelength rangein the IR is of interest.

The filters 20 (FIG. 3a , red/blue) and 27 (FIG. 3d , two types of IR)have in common that spatial interpolation will be required to calculateat each pixel position, a red, blue, first IR, and second IR value. Thefilters 23 (FIG. 3b , green) and 25 (FIG. 3c , IR) do not have thisrequirement, since each pixel is filtered using the same filter.

The present invention is not limited to the example patterned filters ofFIG. 3a-d . What is important is that pattern filter 20 consists of acombination of red and blue filters (no other colours) in a singlepatterned filter, wherein the red and blue filters are arranged in astructure to allow interpolation of the blue values at pixel locationswhere red filters are located and interpolation of the red values atpixel locations where blue filters are located. The 2×2 grouping of redand blue filters as shown in FIG. 3a has an additional advantage in thatwithin the group of identical filters, noise cancelation can beperformed to reduce single pixel noise before interpolation, increasingimage quality. That is, because it can be assumed that pixels in a 2×2grid receive more or less the same light intensity, it is possible toidentify and filter out a values from a malfunctioning pixel.

The grid shown in FIG. 3a , using 2×2 islands of red and blue filteredpixels, is merely an example. For example, it is also possible toalternate red and blue filters in a checkerboard pattern at apixel-by-pixel basis, as in R-B-R-B-R-B- . . . . It is also possible tohave groups of e.g. 1×2 pixels or 2×1 pixels, or any other group size.These considerations do not only apply to the red/blue pattern filter 20of FIG. 3a , but also, mutatis mutandis, to the first and second IRfilter pattern filter 27 of FIG. 3 d.

Some example wavelength ranges for filters are now mentioned. A bluefilter may filter light in the 400-490 nanometre (nm) range. A redfilter may filter 590-660 nm. A green filter may operate in the 490-590nm range. For infrared, the ranges will depend on the application.Generally speaking, there are ranges of interest between 700 and 800 nmand between 800 and 900 nm.

Turning back to FIGS. 1 and 2, in an embodiment of the invention thefilter F3 is a filter 20 (red/blue), F4 is a filter 23 (green), and F5is a filter 25 (IR). Note that the exact mapping to filters F3, F4, F5is not important. What is important for any application is which threefilters are used simultaneously.

In general, the coatings C1, C2, C3 should match the filters F1, F2, F3.For example, the first coating C1 may transmit visible light whilereflecting IR light, so that IR light is guided towards IR filter F3,being an IR filter 25 or 27. The second coating C2 may be transparentfor green light while reflecting red and blue light, so that filter F4should be the red/blue patterned filter 20 and F5 should be the greenfilter 23. These principles for choosing an appropriate coating can begenerally applied to the embodiments disclosed in the description. Forbrevity, in the following embodiments, no more mention will be made ofthe used coatings between prisms.

It is an advantageous aspect of the invention that both the green lightand the IR light in this case do not require interpolation. The red andblue pixels behind the patterned filter 20 of FIG. 3a do require spatialinterpolation. However, the human eye is not as sensitive to details inthe red and blue parts of the spectrum as it is to green. This can forexample be seen from the equation for calculated luminance (luma, L),which reads:L=0.2126*R+0.7152*G+0.0722*B  (1)

Luminance can be seen as the grey-scale representation of visible light,and is most important for distinguishing detail. From this equation itis clear that green light has by far the highest weight. Therefore, itis an advantageous aspect of the present invention that the green lightis not interpolated. At the same time, because of the aboveconsiderations, it is considered an acceptable trade-off that the redand blue parts of the spectrum are spatially interpolated. The IR signalis again not interpolated.

As was already mentioned, an additional advantage of choosing the 2×2filter approach is the possibility of image noise reduction. In imagesproduced by standard Bayer filters, the noise in these visible images isapparent by single pixels highlighting red or blue. This is due to thefact that red and blue need higher gains than green (and thus more noiseis apparent), but because these pixels (when interpolating) have nodirect neighbouring pixels of the same colour, the amount of pixel noisecannot be estimated and therefore not reduced before interpolation. Thisresults in the fact that the interpolation is based on the noise in thepixel and as a result the noise is interpolated as well. It is anadvantageous part of the invention when using the 2×2 patterns, that thegroup of 4 adjacent pixels of the same colour can be used to reducesingle pixel image noise with standard available algorithms beforeinterpolation, resulting in a higher quality image.

In a variant of the above, for applications where two different IRwavelength ranges are to be sampled, the pattern filers 20, 23, and 27(with two different IR filters) can be used for filters F3, F4, F5 inFIGS. 1a and 2. Again, the green light is sampled at full resolution formaximum detail. Red/blue and both IR wavelength ranges are sampled athalf resolution, as a trade-off.

FIGS. 4 and 5 schematically show a dichroic prism assembly according toan embodiment of the invention. In FIG. 4, compared to FIG. 5, the firstpentagonal prism P1 and the first compensator prism P2 are absent. Theresulting dichroic prism assembly of FIG. 3 has four channels, eachrespective channel formed by a path to one of the sensors D2-D5. Thefollowing path lengths are defined for each channel:

-   -   Sensor D2 (e.g. near infrared) path: E+F+G    -   Sensor D3 (e.g. red) path: E+H+I+J+K    -   Sensor D4 (e.g. blue) path: E+H+I+O    -   Sensor D5 (e.g. green) path: E+H+I+M+N

The path lengths are matched, so that E+F+G=E+H+I+J+K=E+H+I+O=E+H+I+M+N.

The following description of the exemplary embodiment will refer to FIG.5, with the understanding that the description of prisms P3-P7 is alsoapplicable to the example of FIG. 4.

In an embodiment of the invention, a pentagonal prism P1 is used. In anembodiment, the incoming light beam A is partially reflected on face S2,being one of the two faces not adjoining the entrance face 51. Thereflected beam B is then reflected against a first one of the facesadjoining the entrance face. The angle of reflection can be below thecritical angle, so that the reflection is not internal (in anembodiment, the adjoining face is coated to avoid leaking of light andreflect the required wavelength of interest). The reflected beam C thencrosses itself (beam A) and exits the prism through the second one ofthe faces adjoining the entrance face, towards sensor D1. A part of thebeam A goes through face S2 and enters compensating prism P2.

In an embodiment according the invention, prism P1 does not use internalreflection to reflect the incoming beam towards sensor D1. In anembodiment, two non-internal reflections are used to direct the incomingbeam A via beams B and C towards sensor D1. In an embodiment there areno airgaps between prisms P1 and P2. In an embodiment, there are noairgaps between prisms P3 and P4. In an embodiment, there are no airgapsbetween prisms P2 and P3.

Prism P2 is a compensator prism which can be moved vertically in orderto adjust the relative beam distances.

In an embodiment of the invention, from P2 the beam D enters a secondpentagonal prism P3. As in prism P1, inward reflection is used to makethe beam cross itself. For brevity, the description of the beam will notbe repeated, except to state that in prism P3, the beam parts E, F, andG correspond to beam parts A, B, and C in prism P1, respectively.

In an embodiment according the invention, prism P3 also does not useinternal reflection to reflect the incoming beam towards sensor D2. Inan embodiment, two non-internal reflections are used to direct theincoming beam E via beams F and G towards sensor D2. In an embodimentthere are no airgaps between prisms P1, P2, P3, and P4.

After prism P3, there is another compensating prism P4. Finally, beam Henters the dichroic prism assembly comprising prisms P5, P6, and P7,with sensors D3, D4, and D5 respectively. The dichroic prism assembly ofP5, P6, and P7 has already been described in reference to FIG. 2. In anembodiment, between prism P4 and prism P5 there is an air gap.

In FIG. 5, the following total path lengths are defined for eachendpoint channel (defined in terms of the sensor at the end of thechannel):

-   -   Sensor D1 (e.g. first near infrared) path: A+B+C    -   Sensor D2 (e.g. second near infrared) path: A+D+E+F+G    -   Sensor D3 (e.g. red) path: A+D+E+H+I+J+K    -   Sensor D4 (e.g. blue) path: A+D+E+H+I+O    -   Sensor D5 (e.g. green) path: A+D+E+H+I+M+N

In an embodiment, the path lengths are matched, so thatA+B+C=A+D+E+F+G=A+D+E+H+I+J+K=A+D+E+H+I+O=A+D+E+H+I+M+N. The matching ofpath lengths can comprise an adjustment for focal plane focus positiondifferences in wavelengths to be detected at the sensors D1-D5. That is,for example the path length towards the sensor for blue (B) light maynot be exactly the same as the path length towards the sensor for red(R) light, since the ideal distances for creating a sharp, focused imageare somewhat dependent on the wavelength of the light. The prisms can beconfigured to allow for these dependencies.

The D+H lengths can be adjusted and act as focus compensators due towavelength shifts.

A larger airgap in path I can be used for additional filters or filledwith glass compensator for focus shifts and compensation. An airgapneeds to exist in that particular bottom surface of red prism because ofthe internal reflection in the path from beam J to beam K. A space canbe reserved between the prism output faces and each of the sensors D1-5to provide an additional filter, or should be filled up with glasscompensators accordingly.

An advantage of the fact that P1 and P3 use essentially the samereflection paths, is that this way the optical assembly can have sensorsmostly on one side (D1, D2, D3 are all on one side), still use an evenamount of direction change so all sensors see the same image and nomirror effects need to be compensated for.

Normally optical prisms are designed as in the case of the three channelblue green and red channels, and that way the path length would requirea very long and particularly wide prism, which would result in a largemodule which is less suitable for use in an endoscope or laparoscope orhandheld imaging system. Furthermore the two channels (sensors D1 andD2) would need to be opposite of each other which also makes theelectronics more cumbersome as the large distance between the 2 sensors.In the embodiment of FIG. 5, the two sensors D1 and D2 can be on thesame side as prisms P1 and P3.

According an embodiment of the invention, the first two prisms P1, P3have a pentagonal shape where at least five corners are used and allcorners have an inside angle >=90 degree.

Each of these prisms has an additional compensator prism P2, P4 tocreate a flat exit surface pane for the light and at the same time havea path length compensation function to be able to match the path lengthsof all sensors.

In an embodiment according the invention, the dimensions are given asfollows. Total height of the module is ≤30 mm or smaller for a 5-channelconfiguration including the sensors. The maximum width ≤15 mm. Themaximum length ≤45 mm.

In an embodiment according the invention, a system using a dichroicprism assembly has a capability to create image resolutions in fullcolour without artefacts at 3840×2160 @ 12 bit per colour at 60 Fpswithout using infrared frame insertion. Infrared frames are available aswell at these same resolution @ 12 bit.

Another advantage of the current invention is that because all surfacesthat connect the paths to D1 and D2 are flat. Accordingly the modules(P1, P2, P3, P4) can be easily aligned which makes automatic assemblyeasy. These modules can be prepared individually and afterwards bondedtogether with a simple robotic or manual tool.

The second prism P3 (for the D2 path) can be smaller than the prism P1(for the D1 path), because path length A is already compensated for. Inorder to shape the first prism for the path to D1 with a slightly longerlength (A) a perfect match can be created between the total internalpath to D1 and compensate as much as possible for the total length ofthe path to D2.

The pentagonal prisms are usually not regular. For example, length a(influencing the length of beams B and C) of the P1 cross section may besmaller than length b (influencing the length of beams A and B).

The above described dichroic prism assemblies of FIG. 4 (with foursensors F2-F5) and FIG. 5 (with five sensors F1-F5) make furtherexemplary embodiments possible, which are below presented schematically:

-   -   an application using two different IR wavelength ranges: the        four-channel dichroic prism assembly of FIG. 4, with for each        respective channel, pattern filters 20 (red/blue), 23 (green),        25 (first IR), and 25 (second IR wavelength range);    -   an application using three different IR wavelength ranges: the        four-channel dichroic prism assembly of FIG. 4, with for each        respective channel, pattern filters 20 (red/blue), 23 (green),        25 (first IR), and 27 (second and third IR wavelength range);    -   an application using four different IR wavelength ranges: the        four-channel dichroic prism assembly of FIG. 4, with for each        respective channel, pattern filters 20 (red/blue), 23 (green),        27 (first and second IR), and 27 (third and fourth IR wavelength        range);    -   an application using three different IR wavelength ranges: the        five-channel dichroic prism assembly of FIG. 5, with for each        respective channel, pattern filters 20 (red/blue), 23 (green),        25 (first IR), another 25 (second IR wavelength range), and yet        another 25 (third IR wavelength range);

Still more combinations can be made using the dichroic prism assembliesof FIGS. 2, 4, and 5 and the pattern filters 20, 23, 25, 27.

As was already mentioned, the present invention is not limited to theexample pattern filters of FIG. 3a-d . The grid shown in FIG. 3a , using2×2 islands of red and blue filtered pixels, is merely an example.

FIG. 6a schematically shows an endoscope 10 or laparoscope 10, accordingto an embodiment of the invention. For the purposes of this invention,the differences between laparoscopes and endoscopes are relativelysmall, so where the description mentions an endoscope, a laparoscopeconfiguration is usually also possible. The endoscope/laparoscope 10comprises camera module 11 including a dichroic prism assembly 15 suchas shown in FIG. 2, 4 or 5. The camera module 11 may also include anadditional lens 12 for focusing incident light onto the entrance face ofthe dichroic prism assembly 15. This last lens 12 may also be integratedin the last part of the endoscope part to match the prism back focallength. An optical fiber 14 connected to a light source 13 couples lightinto the endoscope 10. Inside the endoscope, optical fiber 14 splits offinto several fibers 14′.

Inside the laparoscope a lens system is created consisting of lenselements and/or relay rod lenses (similar as in standard laparoscopes)or, in an endoscope configuration, a fiber bundle can be used like in astandard flexible endoscope. Furthermore a processing unit (not shown)may be placed in the camera module 11 for high speed image processingand image cleanup to optionally reduce data such that it does not haveto be transmitted.

In another embodiment of the invention, the processing unit is placedfurther away from the imaging system and optical elements. A key issuewith specifically endoscopy systems, but also open configurations, isthe cabling which provides the electronic signals of the systems to betransferred between the processing unit and the image sensors placed onthe prism elements, and the cables that provide the lighting to theobject that is imaged. Most configurations have 2 cabling systems basedon Low Voltage Differential Signals (LVDS). The data transfer speed ofthese cables however is mostly limited to the length of the cable. Theoptical configuration proposed in this invention means that more dataneeds to be transferred at a higher bitrate than usual in endoscopicmedical imaging systems. Usability and size are largely hindered by thefact that two cables are required (one for light and one electrical forthe image and control signals). Simply adding more cable pairs is thestandard solution, but the downside of this is that the minimal requiredthickness of the cables is limiting the flexibility and agility of thecamera head. Furthermore the cables will become thicker in diameter andheavier and more electronics is required inside the camera head, whichmakes the camera head larger and heavier.

FIG. 6b shows an embodiment where the dichroic prism assembly 15′ (forexample the assembly shown in FIG. 4 or 5) and the lens 12′ are locatedin the tip of endoscope 10. In this embodiment, there is no need totransport light from the tip to the camera module. However, it isnecessary now to transport the signal from the sensors D1-D5 to themodule 18 housing the processing unit. The rod relaying lenses normallyplaced in the tube of the laparoscope, or the fibers in case of anendoscope are no longer required, and only a simple lens system 12′needs to be placed in front of the prism assembly.

In any of these embodiments, it is possible to include four smalldiameter coax cables 51 and place these around/integrate these with thefiber optic cable 54. The cable is then fused, together with optionalfiller cables 56 to improve stability, as an integrated part and asingle cable, which is directly coupled between camera and controlsystem with light engine. Such an arrangement is shown in FIG. 6c ,where central core fiber 54 can carry light from the light engine to thetip of the endoscope, and the diameter coax cables 51 can carry thesignal from detectors D1-D5 located in the tip to the processing unit inhousing 53. Depending on the amount of data to be transferred and therequired length of the coax cables 51, a suitable number of coax cablescan be selected, e.g. one, two, three, four, or more.

FIG. 7a shows a cross section of an exemplary pentagonal prism for usein a dichroic prism assembly according to an embodiment of the inventionusing the assembly of FIG. 4 or 5.

In an embodiment, the prism P1 or P3 has corners which are designed sothat an incoming beam which enters the entrance face in a directionparallel to a normal of said entrance face is reflected twice inside theprism and exits the prism through an exit face parallel to a normal ofsaid exit face, wherein the normal of the entrance face and the normalof the exit face are perpendicular to each other.

A skilled person can design such a prism. In the example of FIG. 7a ,one of the five angles is a right angle (between faces F1 and F2), twoangles have a first value α≥90 degrees (between faces F2 and F3 and F3and F4) and two angles have a second value 225-α degrees (between facesF1 and F5 and F5 and F4), making a total inside angle of 540 degrees fora pentagonal prism. As can be easily verified, the pentagonal prism asshown in FIG. 7a will inwardly reflect an incoming horizontal beam,incoming through face F1 in parallel with the normal of face F1, usingtwo reflections against faces F3 and F5, so that it exits the prism in avertical direction through face F2, in parallel with the normal of faceF2. In the first reflection, at face F3, the beams will make an anglewith the normal of the face of reflection of α-90 degrees. In the secondreflection, at face F5, the angle with the normal of the face ofreflection is 135-α. In this example, the angle α should have a value90<α<135.

In a specific embodiment, the value of a is 112.5 degrees, so that thepentagonal prism has one right angle and four angles of 112.5 degrees.

FIG. 7b shows a reduced part of the prism of FIG. 7a . As can be seenfrom the construction of the lines, it is sufficient if one angle in theprism is 90 degrees (between F1 and F2), an angle adjoining the 90degrees angle is a, and the other angle (between F1 and F5) adjoiningthe 90 degrees angle is 225-a. The angles between F3 and F4 and F4 andF5 can in fact be freely chosen.

The skilled person will also be able to design prisms in which theincoming and outgoing beams are not at right angles to each other.However, there are practical advantages to designs, such as theembodiment of FIG. 7b , wherein the incoming and outgoing chief ray beamare at a right angle to each other, as additional angles of light arecoming from the lens system.

A pentagonal prism shape is a logical and practical choice for theprisms P1 and P3. However, as is clear from FIG. 7b , in order to createthe conditions for the required reflections, in fact many differenttypes of prism shape will serve. The invention is therefore not limitedto the use of pentagonal prisms.

In the foregoing description of the figures, the invention has beendescribed with reference to specific embodiments thereof. It will,however, be evident that various modifications and changes may be madethereto without departing from the scope of the invention as summarizedin the attached claims.

In addition, many modifications may be made to adapt a particularsituation or material to the teachings of the invention withoutdeparting from the essential scope thereof. Therefore, it is intendedthat the invention not be limited to the particular embodimentsdisclosed, but that the invention will include all embodiments fallingwithin the scope of the appended claims.

In particular, combinations of specific features of various aspects ofthe invention may be made. An aspect of the invention may be furtheradvantageously enhanced by adding a feature that was described inrelation to another aspect of the invention.

It is to be understood that the invention is limited by the annexedclaims and its technical equivalents only. In this document and in itsclaims, the verb “to comprise” and its conjugations are used in theirnon-limiting sense to mean that items following the word are included,without excluding items not specifically mentioned. In addition,reference to an element by the indefinite article “a” or “an” does notexclude the possibility that more than one of the element is present,unless the context clearly requires that there be one and only one ofthe elements. The indefinite article “a” or “an” thus usually means “atleast one”.

The invention claimed is:
 1. A method for detecting visible and infraredlight using a medical imaging system, said medical imaging systemcomprising a dichroic prism assembly and a camera module configured toreceive visible light and infrared light from an object image, themethod comprising: exciting a fluorescence agent in a tissue sample withexcitation light; illuminating the tissue sample with visible light;receiving light from the object image through an entrance face of thedichroic prism assembly, said light comprising visible light and one ormore infrared fluorescence light originating from the fluorescenceagent; splitting the received light into a first, second, and thirdoptical path; filtering the received light in the first, second, andthird optical path through a first, second and third optical filter,respectively; detecting the filtered light in the first, second, andthird optical path with a first, second and third sensor, respectivelywherein the first, second, and third filters, in any order, are a greenfilter, an infrared filter, and a red/blue patterned filter comprisingred and blue filters in alternating pattern so that half of the lightreceived by the red/blue patterned filter goes through a blue filter andhalf of the light received by the red/blue patterned filter goes througha red filter, wherein the dichroic prism assembly comprises at least afirst, second, and third prism, each prism having a respective first,second, and third exit face, wherein: the first exit face is providedwith the first sensor, the second exit face is provided with the secondsensor, and the third exit face is provided with the third sensor, andwherein the dichroic prism assembly comprises a fourth prism, saidfourth prism having a cross section with at least five corners, eachcorner having an inside angle of at least 90 degrees, the fourth prismproviding a fourth optical path with a fourth filter and having a fourthexit face provided with a fourth light sensor, wherein the fourth filteris a second infrared filter.
 2. The method according to claim 1, whereinthe red/blue patterned filter comprises groups of a plurality of pixels,each group having a filter for either red or blue light.
 3. The methodaccording to claim 2, wherein the groups having different filters arearranged in a checkerboard pattern.
 4. The method according to claim 1,wherein the infrared filter is transparent for light in an infraredwavelength range, said range starting above 700 nm and ending below 800nm or starting above 800 nm and ending below 900 nm.
 5. The methodaccording to claim 1, wherein the infrared filter comprises groups ofone or more pixels, each group having a filter for either a first orsecond infrared wavelength range, wherein the groups having the first orsecond infrared wavelength range are arranged in an alternating pattern,such as a checkerboard pattern.
 6. The method according to claim 5,wherein the first wavelength range starts above 700 nm and ends below800 nm and the second wavelength range starts above 800 nm and endsbelow 900 nm.
 7. The method according to claim 1, further comprising: inaddition to the splitting of the received light into a first, second,and third optical path, splitting the received light into a fourthoptical path, and filtering the received light in the fourth opticalpath through a fourth optical filter, wherein the fourth optical filteris a second infrared filter.
 8. A medical imaging system for detectingvisible and infrared light, said medical imaging system comprising aninput for visible light for illuminating a tissue; an input forexcitation light for exciting a fluorescence agent in the tissue; acamera module configured to receive visible light and infrared lightfrom an object image in the tissue, the camera module comprising atleast a first, second, and third optical path for directing light fromthe object image to a first, second, and third sensor respectively, anda dichroic prism assembly configured to receive the light from theobject image through an entrance face of the dichroic prism assembly,comprising at least a first, second, and third prism, each prism havinga respective first, second, and third exit face, wherein: the firstoptical path is provided with a first filter, the second optical path isprovided with a second filter, and the third optical path is providedwith a third filter, wherein the first, second, and third filters, inany order, are a green filter, an infrared filter, and a red/bluepatterned filter comprising red and blue filters in alternating patternso that half of the light received by the red/blue patterned filter goesthrough a blue filter and half of the light received by the red/bluepatterned filter goes through a red filter, wherein: the first exit faceis provided with the first sensor, the second exit face is provided withthe second sensor, and the third exit face is provided with the thirdsensor, and wherein the dichroic prism assembly comprises a fourthprism, said fourth prism having a cross section with at least fivecorners, each corner having an inside angle of at least 90 degrees, thefourth prism providing a fourth optical path with a fourth filter andhaving a fourth exit face provided with a fourth light sensor, whereinthe fourth filter is a second infrared filter.
 9. The medical imagingsystem according to claim 8, wherein the red/blue patterned filtercomprises groups of a plurality of pixels, each group having a filterfor either red or blue light.
 10. The medical imaging system accordingto claim 9, wherein the groups having different filters are arranged ina checkerboard pattern.
 11. The medical imaging system according toclaim 8, wherein the infrared filter is transparent for light in aninfrared wavelength range, said range starting above 700 nm and endingbelow 800 nm or starting above 800 nm and ending below 900 nm.
 12. Themedical imaging system according to claim 8, wherein the infrared filtercomprises groups of one or more pixels, each group having a filter foreither a first or second infrared wavelength range, wherein the groupshaving the first or second infrared wavelength range are arranged in analternating pattern, such as a checkerboard pattern.
 13. The medicalimaging system according to claim 12, wherein the first wavelength rangestarts above 700 nm and ends below 800 nm and the second wavelengthrange starts above 800 nm and ends below 900 nm.
 14. The medical imagingsystem according to claim 8, comprising a fifth prism, said fifth prismhaving a cross section with at least five corners, each corner having aninside angle of at least 90 degrees, the fifth prism providing a fifthoptical path with a fifth filter and having a fifth exit face providedwith a fifth light sensor, wherein the fifth filter is a third infraredfilter.
 15. Endoscope or laparoscope or open lens system comprising atleast one medical imaging system according to claim
 8. 16. The endoscopeor the laparoscope or the open lens system according to claim 15,further comprising a lens for focusing light onto the camera module. 17.The endoscope or the laparoscope or the open lens system according toclaim 15, using an integrated cable comprising an optical fiber core forproviding light from a light engine onto a surgical field to illuminate,the optical fiber core surrounded by a plurality of coax cables fortransporting sensor data.
 18. The endoscope or the laparoscope or theopen lens system according to claim 15, wherein a frame acquisitionfrequency is at least 60 frames per second and a sampling rate is atleast 8 bits per pixel.